The present invention relates generally to databases and more specifically to methods to access large key-value stores.
A key-value store uses an associative array (also known as a map or dictionary) as its data model. A key-value store is also known as a key-value database. Data is stored as a collection of key-value pairs, such that each possible key appears at most once in the collection of key-value pairs. The value is data that is accessed by a client by presenting the key-value store with the key that is associated with the data. The data may consist of a plurality fields. The structure of key-value stores are amenable to use in large databases that are distributed across a plurality of computer systems, because they lend themselves to scaling (accommodating increasing numbers of key-value pairs) and partitioning (being divided up into manageable pieces).
Often database clients access a key-value store over a network that uses a network protocol such as TCP/IP. The processing of a general purpose network protocol is involved and can create a bottleneck on a server that is hosting a key-value store for a plurality of clients. Such a bottleneck can constrain and limit the performance of the key-value store.
RDMA (Remote Direct Memory Access) is the name given to a technique to access (i.e. read from or write to) a memory location on a remote system without interrupting the processing of the CPU(s) on that system. It is therefore time and resource efficient. A RDMA access uses information (e.g., pointers) returned from the remote system when memory in the remote system is initially accessed. Subsequent accesses to the same memory use the returned information to facilitate the access.
RDMA provides a messaging service, which applications can use to directly access the virtual memory on remote computers. The messaging service can be used to communicate with storage devices on a remote computer directly using several well-defined Upper Layer Protocols (ULPs), e.g., SCSI RDMA Protocol, and bypassing the operating system on the remote computer. Bypassing the operating system on a system can enable fast access to the data on the system.